feat(runtime): add Plugin Platform foundation - #3729

Addressed all four findings in 580a09982.

• Install rollback outcome: Package rollback no longer swallows removal, restore, or directory-sync failures. Any rollback that cannot confirm a durable directory state returns commit_outcome_unknown, retains recoverable transaction remnants, and immediately fences the Plugin Platform.
• Fence admission: mutable state is re-checked inside the serialized callback. A same-tick operation that queued before another operation established a fence can no longer execute afterward. Added a regression that holds the first commit, queues the second mutation, establishes an unknown outcome, and verifies the second never reaches persistence.
• Post-commit output limit: removed the speculative full-state plugin.composition.apply and plugin.platform.query responses. The public slice now exposes only fixed, bounded install/uninstall results, so a successful durable commit cannot fail later while encoding a materialized Desired Tree or Inspection projection.
• Concrete consumer and scope: added maka runtime-host plugin install|uninstall as the end-to-end consumer. Removed the unused public reload, export, generic composition apply, full query, Runtime projection/digest, and bundle-export surfaces. The public protocol is now two lifecycle operations instead of six, and the PR changed from +4859/-64 to +4528/-78.

Verification: full workspace build, Biome, ASF headers, and diff check passed; Runtime 3037/3037 active tests and CLI 455/455 passed; Plugin Platform + CLI focused coverage is 25/25. Runtime Host passed 1195/1196 in the parallel run; the sole unrelated shared registration-directory race passed 3/3 isolated reruns.

Reworked the current head against the latest review in 85b5aac09:

• Package replacement now uses a durable generation journal. Candidate bytes are validated first; canonical bytes and Composition authority commit before Runtime convergence. Recovery restores the previous Package at the base generation and retains the new Package at or beyond the committed generation.
• Unknown Package/Composition outcomes fence the platform, and every queued mutation re-checks the fence when dequeued.
• Uninstall plans and validates the candidate authority before committing, so package_in_use leaves layers and Desired state unchanged.
• plugin.composition.apply now returns only the committed generation. plugin.platform.query remains available through bounded status/packages/entries/failures views with byte-bounded cursor pages.
• All six operations now have concrete maka runtime-host plugin ... CLI consumers; Bundle export remains supported. The unused Runtime digest/projection contract was removed.
• Close attempts both Runtime composition and generation-loader cleanup and aggregates failures.

Added crash-boundary, queued-fence, uninstall, response-size, close-aggregation, and CLI consumer regressions. Full Runtime (3037), Runtime Host (1198), and CLI (455) suites pass, along with full build, Biome, ASF, Windows inventory, and diff checks.

Follow-up on the three prototype-key P3s from the review body: fixed in 0239008. (1) applyCompositionState now creates a missing session scope with an own data property, so session:proto is preserved without invoking Object.prototype.proto or throwing from siblings.splice. (2) scalar/isolate decoders build records with Object.fromEntries; valid prototype-named scalar keys survive, while an invalid proto service key is now rejected rather than silently disappearing. (3) Extension configuration reads only own input properties, so declared constructor/toString/valueOf-style keys no longer resolve inherited values. Focused reducer, protocol, and manifest tests were added. Runtime: 3051 tests, 0 failures (13 skipped). Runtime Host: 1207/1207. Biome, ASF headers, and diff check pass.

State-machine review: make the hidden product state explicit

I reviewed the current head (023900813) as a state-coordination protocol rather than as a collection of Plugin APIs. The package transaction fixes on this head materially improve the crash boundaries. The remaining concern is now one level higher: several correctness properties depend on state that exists only implicitly in call order, private booleans, temporary journal layout, or operator knowledge.

The plain-language model

Imagine the Host has four things:

1. A locked cupboard of Plugin boxes — the Package Store. Each box contains code, a manifest, and possibly a Composition patch.
2. An official ledger — the persisted Composition authority. It records an epoch, the ordered Package layers, and the User overlay operations.
3. A written plan reconstructed from the cupboard and ledger — the Desired Entry Tree.
4. The machines currently running in the house — the live Context/Fiber Runtime.

The important rule is that these are not the same state:

P = Package bytes, manifests, patches, and Runtime modules
A = persisted authority: epoch + packageLayers + overlays
D = derive(P, A): the Desired Entry Tree
L = converge(D, code(P)): the live Fiber Tree
Q = inspect(L): the operator-facing projection

(P, A) -> D -> L -> Q

The official facts are jointly P + A. D is an in-memory projection. L is a fallible execution projection. Q is only an observation of that execution projection.

This distinction explains the otherwise unusual mutation order:

install:   publish Package bytes -> commit authority -> converge Runtime
uninstall: commit authority      -> detach Runtime  -> remove Package bytes
apply:     commit authority      -> converge Runtime

Installation must put the box in the cupboard before the ledger is allowed to reference it. Uninstallation must remove the ledger reference before throwing the box away. Extra, unreferenced bytes are a safe orphan; an authority that references missing bytes is not safe.

Package replacement adds a recoverable receipt:

prepare candidate
  -> fsync candidate
  -> journal baseEpoch -> nextEpoch
  -> publish candidate as canonical Package
  -> commit Composition authority at nextEpoch
  -> remove transaction remnants
  -> converge live Runtime

After owner death, the persisted authority is the transaction decision record:

authority == baseEpoch  -> restore the previous Package
authority >= nextEpoch  -> retain the candidate Package
anything else           -> the evidence is ambiguous; fence

That ordering is the strongest part of this PR. The Plugin Platform is effectively a transaction coordinator for two durable stores followed by a Runtime reconciler.

This is a product of state machines, not one state machine

At minimum, the implementation has these independent axes:

Control phase:
  NEW | RECOVERING | READY | FENCED | DRAINING | CLOSED

Convergence:
  UNKNOWN | CONVERGED | DIVERGED

Package transaction:
  ABSENT | PREPARED | PUBLISHED_UNCOMMITTED | COMMITTED | ROLLING_BACK

Authority commit:
  OLD | COMMITTING | COMMITTED | OUTCOME_UNKNOWN

Live Runtime:
  OLD | STAGING | CONVERGED | PARTIALLY_RECOVERED | FAILED

Today those states are encoded indirectly by #closed, #draining, #poisoned, #diverged, filesystem layout, authority generation, and whether a particular method has already been called. Only a subset of the possible combinations is valid, but the type/API boundary does not say which subset.

The state-machine review question is therefore not merely “does each method work?” It is:

For every externally observable transition, can the implementation prove from durable evidence whether the command was rejected, committed, committed-but-not-converged, or ambiguous — and can the caller observe that distinction without guessing?

On the current head, several answers still depend on hidden state.

Blocking implicit state dependencies

1. recover() must happen exactly once, but this lifecycle state is not represented

HostPluginPlatform starts with an empty in-memory authority and immediately permits installPackage, uninstallPackage, apply, and read as long as it is not closed, draining, or poisoned. There is no NEW/READY admission check.

Production composition currently calls recovery before exposing ready operations, but the class is public and the invariant is not enforced by the class. Calling a mutation before recover() can plan from generation 0 and an empty layer list even when durable authority already exists.

The inverse transition is also unguarded: calling recover() a second time on a live instance runs packageLoader.collectGarbage() before rebuilding. That can remove executable-generation directories still owned by live Fibers. The review response says recovery is intentionally one-shot; the state machine should encode and reject the unsupported transition rather than rely on caller knowledge.

Required model:

NEW -> RECOVERING -> READY | FENCED
READY -/-> RECOVERING
NEW -/-> mutation/query

2. A known durable commit is erased at the protocol boundary

HostPluginPlatform.apply() and installPackage() correctly distinguish a post-commit Runtime failure internally. They set #diverged and throw HostPluginPlatformError('mutation_failed', '... authority was committed but Runtime convergence failed').

HostPluginPlatformCoordinator.failure(), however, unwraps every mutation_failed to its cause:

if (error.code === 'mutation_failed' && error.cause) return failure(error.cause);

The wire caller can therefore receive an ordinary internal_failure or another cause-derived error for two fundamentally different outcomes:

A. rejected before commit; durable state did not change
B. authority committed; Runtime did not converge

Retrying B as though it were A is unsafe. A retry can append another overlay, hit entry_exists/entry_not_found, or turn an install acknowledgement retry into a replacement.

This is not commit_outcome_unknown; the outcome is known to be committed. It needs a first-class wire result, for example:

{ ok: true, result: { generation, convergence: 'diverged', failures } }

or a distinct committed_runtime_failed outcome carrying the committed generation. The coordinator must not discard the commit fact.

3. #diverged changes the transition algorithm but is not observable

#diverged is not a diagnostic-only bit. It decides whether a later apply uses incremental composition.apply() or full recoverComposition(), and whether reload triggers reconciliation. It is therefore part of the product state machine.

The status query does not expose it. Instead it combines:

generation   <- Desired authority
packageCount <- physical Package Store
entryCount   <- live Runtime
failureCount <- ephemeral in-memory failure projection

During divergence, status may report the new authority generation, the old live Entry count, and zero failures. In particular, a replacement can fail during composition.reload() after authority commit but before #publishEntryFailures() records Entry failures.

Status should expose at least:

{
  phase,
  authorityEpoch,
  convergence: 'converged' | 'diverged',
  desiredEntryCount,
  liveEntryCount,
  installedPackageCount,
  layeredPackageCount,
}

There is also no active reconciliation loop. Divergence is repaired only by a later mutation/reload or by restart, so an idle Platform can remain diverged indefinitely. That trigger dependency must be explicit, or a plugin.platform.reconcile operation/background policy must own it.

4. The authority is composite, but no durable Package revision is recorded

The persisted authority stores Package IDs, not Package revisions:

{ generation, packageLayers: string[], overlays }

The Desired Tree is not determined by that file alone. It also depends on the current manifest and patch bytes under each canonical Package directory. The install journal temporarily binds “old bytes” and “candidate bytes” to baseGeneration -> nextGeneration, but after transaction cleanup the authority retains no version, digest, or patch identity.

Therefore the real durable authority is (Composition file, canonical Package bytes), under the implicit assumption that no path except PluginPackageStore mutates those bytes.

That may be the intended trusted-control-directory contract, but it should be explicit and inspectable. A stronger authority would reference an immutable revision/digest:

{ extensionId, packageRevision, layerPosition }

The Package query should expose the revision as well. Otherwise an operator can identify the Package ID but cannot prove which code/patch revision produced the current Desired Tree.

5. Serialization is part of correctness, but bypass paths are public

Between Package publication and authority commit, canonical Package bytes and authority intentionally disagree. The design is safe only because:

• every query and mutation uses the same serialized queue;
• recovery runs before the Host serves requests;
• no other writer mutates either store;
• no caller reads the canonical Package directly during the transition.

Those are transaction invariants, not implementation conveniences.

However PluginPackageStore, HostPluginCompositionStore, and their direct mutation methods are exported. PluginPackageStore.install() can publish with hard-coded 0 -> 1 without coordinating Composition authority. A caller can therefore bypass the only coordinator that makes the two stores consistent.

If HostPluginPlatform is the sole coordinator, lower-level mutation capabilities should be internal or explicitly marked unsafe/test-only. Read-only inspection can remain public; cross-authority writes should not.

6. Package patches create an undeclared dependency graph

The manifest declares explicit Package dependencies, and the Platform validates their presence, cycles, same-root activation, and uninstall protection.

But ordered Composition patches create a second dependency channel. Package B can update, move, disable, or remove an Entry introduced by Package A. B's patch can only replay if A's Entry already exists, so B has a structural dependency on A even when B's manifest declares none.

This hidden edge affects:

• installation order;
• replacement validation;
• startup replay;
• uninstall planning;
• operator explanations.

For example, uninstalling A can make replay of B fail on entry_not_found, which means A is effectively in use by B, but Package query exposes only manifest dependencies.

The model needs one of these explicit contracts:

1. patch references generate dependency edges and become queryable;
2. a Package patch may mutate only Entries owned by that Package;
3. cross-Package overrides require an explicit extends/overrides declaration.

Without one, packageLayers is simultaneously precedence order and an invisible dependency graph.

7. Online derivation and recovery derivation are two algorithms

Online planning uses #composeLayers(): normalize each layer, apply it, normalize overlays, then validate strictly before commit.

Startup uses #composePersistedAuthority(): replay raw layers and overlays, normalize the completed tree, then isolate invalid Entries during recovery.

The strict-versus-fail-open policy difference is legitimate. The implicit dependency is that both implementations must forever derive the same valid tree despite different ordering and validation paths.

This should be one projector with explicit policy:

deriveDesired({
  authority,
  packageResolver,
  validation: 'strict' | 'isolate-invalid',
}) -> { desired, failures, dependencyGraph }

Then online mutation and startup recovery differ by policy rather than by duplicated causal algorithms.

8. Three unrelated concepts are all named generation

The system currently has:

1. Composition authority generation — the durable transaction epoch;
2. Fiber generation — the activation instance shown in Entry inspection;
3. executable Package generation — the immutable UUID directory holding code.

They are not comparable, but they share the same word in the same protocol/domain. baseGeneration/nextGeneration refer to authority; Entry generation refers to Fiber activation; plugin-generations-v1 refers to code revision.

These should be distinct types and names, such as:

authorityEpoch
fiberActivationId
packageRevision

Otherwise future code will eventually compare or report the wrong generation.

9. Paged queries depend on a stable snapshot that is not represented

The cursor is an array offset. Each page is serialized, but separate page requests can have mutations between them. Packages, live Entries, and failures can be inserted, removed, or reordered, causing an offset cursor to skip or repeat items.

The cursor needs to bind to a projection revision/snapshot token, or the request must carry an expected authority/live projection epoch and reject when it changes.

10. The contribution state machine is not connected in production yet

Production constructs new HostPluginPlatform(controlDirectory), which creates a default MakaCompositionLoader() with no Runtime Host root Context and no concrete Tool/UI/Hook/Event registry transaction. MakaPluginPackage.contributions is validated and retained, but there is no product contribution consumer in this PR.

The implemented causal chain therefore currently ends at:

Package -> Desired Entry -> Context/Fiber

not yet:

Package -> Tool/UI/Hook/Event contribution -> product behavior

That missing edge matters to state correctness. Make-before-break reload currently stages a new Fiber while the old Fiber still owns global Service labels; a Service-providing Plugin can collide before the old owner retires. Contribution publication, rollback, partial recovery, and external registry ownership need at least one concrete consumer test before the Platform can claim end-to-end convergence semantics.

State invariants I would make executable

The following should be named invariants with operation-boundary and crash-boundary tests:

I1. READY implies recovery completed exactly once.
I2. authorityEpoch == desired.authorityEpoch in every settled in-memory state.
I3. derive(P, A) == D, or the Platform is FENCED with durable evidence explaining why.
I4. L matches D, or convergence == DIVERGED and that state is externally observable.
I5. A reference to a Package implies the referenced Package revision exists.
I6. A pre-commit rejection is distinguishable from a committed Runtime failure.
I7. An unknown durable outcome fences every later mutation at dequeue time.
I8. Every dependency that constrains replay/uninstall is present in one queryable graph.
I9. Every public page belongs to a stable projection snapshot.
I10. No public lower-level write can bypass the cross-store coordinator.

Bottom line

The crash-recovery mechanics are now close to coherent. The main remaining risk is that the control plane does not yet expose the same state distinctions that the implementation relies on internally.

The most important fixes before merge are:

1. preserve “committed but Runtime diverged” at the wire boundary;
2. expose convergence and desired/live projections in status;
3. encode NEW -> RECOVERING -> READY/FENCED and reject unsupported lifecycle transitions;
4. bind authority to an inspectable Package revision;
5. remove public cross-store write bypasses;
6. make patch-reference dependencies explicit;
7. unify Desired derivation behind one policy-driven projector.

Once those states and dependencies are explicit, the Package journal, authority commit, and Runtime reconciliation can be reasoned about as one defensible system rather than as a set of methods whose correctness depends on callers already knowing the hidden state machine.

Implemented the state-machine review on the rebased head 39937b5d7 (base 8b93dd52b). This supersedes my earlier scope objections on the affected threads.

1. Recovery lifecycle: HostPluginPlatform now enforces one-shot recovery, rejects mutation/query before readiness, and keeps recovery completion independent from drain/close so production shutdown remains safe.
2. Known commit outcome: post-commit Runtime failures return a committed mutation receipt with authorityEpoch and convergence: diverged; the coordinator no longer unwraps away the commit fact.
3. Observable convergence: status exposes phase, authority epoch, convergence, physical/layered Package counts, desired/live Entry counts, failures, and a bounded fence diagnostic. Divergence gets both explicit reconcile and a bounded backoff reconciler.
4. Package identity without restoring Revision: the authority remains canonical Package bytes + Package IDs/layers + overlays. Package query exposes a computed canonical contentDigest for inspection only; no Package Revision/Version/Binding state was reintroduced.
5. Single coordination boundary: direct Store mutation is removed and Store/Loader implementations are no longer public server exports; the Platform privately owns all cross-Store transitions.
6. Cross-Package patches: manifests explicitly declare composition.structuralDependencies; the shared projector verifies the actual cross-Package structural edges, query exposes them and reverse requiredBy, and uninstall protects both manifest and structural dependents.
7. One Desired projector: online planning and recovery now use the same derivation implementation with strict versus recovery policy parameters.
8. Stable pagination: cursors are opaque and snapshot-bound (view/root/digest/offset); a mutation between pages produces stale_cursor rather than silent duplicates/omissions.
9. Generation terminology: public Platform receipts/status use authorityEpoch. Computed Package identity is contentDigest; Fiber inspection keeps its nested Runtime generation. The composition baseGeneration field remains only the existing CAS input.
10. Real consumer boundary: the PR description now says Package -> Desired Entry -> Context/Fiber, not generic contribution convergence. No Tool/UI/Hook/Event consumer was added. The concrete single-provider Service collision is fixed with break-before-make Package replacement and covered by a real provider regression.

Also changed Plugin Platform admission to local-owner-only, made committed uninstall cleanup failure caller-visible as cleanup: pending, and added loaded-generation release on pre-adoption failures.

Verification on the pushed head: complete workspace build; Runtime Host 1390 passed / 9 skipped / 0 failed; focused Plugin Platform 35/35; focused CLI 7/7; Biome and git diff --check pass.